Add setup_environment to truss
#1188
Conversation
| @@ -298,6 +298,12 @@ def _run_docker(gpus: Optional[str] = None): | |||
| f"src={str(LocalConfigHandler.bptr_data_resolution_dir_path())}", | |||
| "target=/bptr", | |||
| ], | |||
| [ | |||
There was a problem hiding this comment.
for my own edification, what does this help accomplish?
There was a problem hiding this comment.
we mimic the mounted configmap directory (which is /etc/b10_dynamic_config on model pods) here so we can make changes to /etc/b10_dynamic_config/environment in the integration tests to mimic patches to the environment key in the dynamic_config configmap
spal1
force-pushed
the
samiksha/bt-12427-add-support-for-setup_environment-to-truss
branch
2 times, most recently
from
59a4aa9
to
3bae49a
Compare
|
|
||
| # Skip polling if no setup_environment implementation provided | ||
| if not self.model_descriptor.setup_environment: | ||
| self._logger.info("No model.setup_environment definition provided") |
There was a problem hiding this comment.
I don't think we need to log here, if the user didn't provide a setup_environment method, I don't think this is the right way to communicate this to them.
On the other hand, if they did provide a method, it might be good to log something indicating that we will poll for changes to the env.
| @@ -96,6 +97,7 @@ pytest = "7.2.0" | |||
| pytest-cov = "^3.0.0" | |||
| types-PyYAML = "^6.0.12.12" | |||
| types-setuptools = "^69.0.0.0" | |||
| types-aiofiles = "^24.1.0.20240626" | |||
There was a problem hiding this comment.
can we use a less specific version here? (maybe "^24.1.0")
| await self.setup_environment(environment_json) | ||
| self._environment = environment_json | ||
| except Exception as e: | ||
| logging.error( |
There was a problem hiding this comment.
I think it shows up nicer if you do:
logging.error("blah blah", exc_info=e)
🚀 What
This PR adds a new
setup_environmentfunction that we will call when we notice a change to the environment a truss is running in. This environment refers to the release environment, ex:productionorstaging. In order to notice these changes, we will need to watch for updates to a particularenvironmentfile mounted on theb10_dynamic_configdirectory.Going to add support for
setup_environmentfor chainlets in a separate PR: #1192💻 How
on_startup, we call a `setup_polling_for_environment_updates() function that will create the asyncio task that will run the file watcherModelWrapper._statusto beREADY, indicating that theload()was successfully completed, and then will start actually watching the/etc/b10_dynamic_config/environmentfiledynamic_config_resolverto access the contents of the/etc/b10_dynamic_config/environmentfile and call the user-definedsetup_environmentfunction with the contents🔬 Testing
TODO:
setup_environmentfromload_impl- this is to enable setting up the environment before user-definedloadis calledtruss/tests/test_model_inference.pydynamic_config_resolverLoom with e2e test: https://www.loom.com/share/cbe8e03a7d97434f99f973eaceb33c59?sid=ca51f099-fd45-4b39-a7bf-366d7d01356b